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A STUDY OF GRATEFUL MED USE IN A GRADUATE HEALTH PROGRAM 
ABSTRACT 

A study of GRATEFUL MED use by 68 students enrv^ilea in graduate 
level courses at California State University Northridge showed that 
97% of the students liked searching MEDLINE using this front-end 
software. The classes, the instructional methods, and the research 
methodology are described. Data collected from students' 
evaluation forms indicate in-depth training including hands-on use 
of the software gave highest level of satisfaction. Students who 
had previous experience with computers felt the most: comfortable 
with the software. Ovar 75% of the students needed literature 
published before 1986. Many of the students planned to obtain 
their own National Library of Medicine (NLM) access codes and 
Grateful Med software. Recent experience shows students ' 
preference for using MeSH on DIALOG'S CD-ROM. Future studies will 
examine students' ability to use both Grateful Med and MEDLINE CD- 
ROM for exhaustive research. 
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In the last few years end-user searching has become increasingly 
popular* The host of new information technologies offers an 
interesting challenge in library instruction. This is especially 
true when instructing graduate level classes where students usually 
have an immediate, exhaustive research need, as well as a need to 
learn how to search on system(s) which can be used once they leave 
the university environment. Several recent articles on end-user 
searching [1,2,3,4] describe the various online sources f*^r the 
MEDLINE databases. The system chosen for student use was National 
Library of Medicine's (NLM's) reduced rate student program and the 
inexpensive, well-reviewed [5,6,7] front-end, telecommunication 
software. Grateful Med. 

BACKGROUND MUD PURPOSE OF STUDY 

A study of Grateful Med use by California State University 
Northridge(CSUN) students was conducted Spring 1988; this study was 
funded by a CSUN Faculty Affirmative Action Grant of $500. The 
purpose of the study was to investigate Grateful Med software's 
potential for students. CSUN had a long tradition of offering 
librarian mediated MEDLINE searches on a cos+- recovery basis. 
When first introduced, MEDLINE searching with Grateful Med software 
was the only end-user program CSUN library had for searching the 
professional, medical literature. Although the library had begun 
to acquire some CD-ROM products, funding for a range of datalases 
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plus the necessary equipment was a slow process. A study of 
Jrateful Med was a good opportunity to study the use of a system 
which would also be offering the students lifelong skills. We 
specifically wanted to find out how efficiently students can access 
MEDLINE using Grateful Med, what the average costs would be, and 
how frequently students needed access to the backfiles. The 
students chosen for the study were health science and communicative 
disorders students enrolled in research methods classes. All 
students were given one page of searching instructions and one page 
of order information for Grateful Med purchase and for acquiring a 
personal student access code. Those students v/ho returned to use 
the library's funded access code were asked to complete a two-page 
evaluation form. Although the Grateful Msd software was 

demonstrated to eight classes, only four classes, (seventy three 
students) , received instruction with the expectation that many of 
these students would return to use the library's software and 
funded access code. 

Sixty-eight evaluation forms were returned. Data collected on 
three variables (cost, relevant retrieval, user satisfaction) were 
correlated to the type of instruction received as well as previous 
computer experience of the patron. 

CLASS DESCRIPTIONS 

All of the classes received instruction on the structure of Medical 
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Subject Headings (MeSH) and on basic principles of specific concept 
indexing. All students were urged to consult MeSH first, and then 
to check the printed Index Medicus before going online. All 
students were encouraged to select search terms, whenever possible, 
from Grateful Med's MeSH. The use of the <F10> function key was 
clearly emphasized in all classes. This approach, emphasizing 
controlled vocabulary, continued the traditional instruction for 
the use of printed Index Medicus . Considering the large size of 
the classes, and the brief amount of available instructional time, 
students needed to learn the most efficient way to retrieve 
relevant citations. Additional instructional methods and 
evaluating relevancy were described in recent articles on Grateful 
Med use [8,9,10] and on MEDLINE end-user training [10-12]. 

The four classes received various leveT.G of instruction. The first 
class to receive instruction on Grateful Med was the Research 
Methods Class in Communicative Disorders. The proj.essor wanted a 
2 1/2 hour lecture covering all phases of library research. The 
class received a ten minute demonstration of Grateful Med software 
using a 19" Sony color monitor and an IBM XT with a 1200 baud 
modem- The professor's assignment was to prepare bibliographies on 
two topics using both the printed index and MEDLINE using Grateful 
Med. Since this lecture did not emphasize the mechanics of the 
Grateful Med software, the librarian was available when the 
students used the software Cor further individualized instruction. 
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The class of thirty had a deadline of one week, this meant that 
thirty students, researching two topics each, had to schedule 
computer time within a seven day interval. 

The second class, a Research Methods Class for physical therapy 
students, received a two-hour lecture. For this class, arrangements 
were made with the computer center to use the microcomputer 
laboratory where students could actually use the Grateful Med 
software to formulate a search. Since the PCs did not have modems, 
students only experienced the input of the search strategy. The 
first eighty minutes were given in the same classroom where the 
Communicative Disorders Research Methods class received their 
instruction. The same equipment for the online demonstration was 
used. The last half hour was given in the microcomputer 

laboratory so students would have hands-on experience. Although 
instruction on Grateful Med was given to the entire class of thirty 
nine students, only a small fraction of those were expected to 
return to use the software installed on the PC with modem for 
student use. The class had been divided into topic groups and only 
one person from each group was to schedule computer time. However, 
the entire class exhibited strong interest in learning how to use 
Grateful Med. Only one student chose not to participate in the 
second session in the microcomputer lab. This meant, however, that 
each PC had to be shared by two students. Group topics were t e 
semester's research problem. The literature search was just one 



phase and had no immediate deadline. 
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The third class, a Research Methods Class for graduate health 
science students, was given a 2 1/2 hour lecture at an off -campus 
site. The classroom was not equipped for demonstrations so the 
lecture was done using an overhead projector for examples. Again, 
the structure of MeSH, waj emphasized. Since these graduate 
students do not have a library available at the campus where they 
take class ".s, the classroom instruction did not include an overview 
of a variety of printed indexes and abstracts that would be 
relevant. The entire time was devoted to searching for information 
online, mostly searching MEDLINE using Grateful Med, but other 
online services were mentioned. The facility had two PC clones 
with modems. The class was advised that they would need to bring 
floppy disk(s) to download searches. Two sets of disks with 
separate passv/ords v/ere left as well as a copy of MeSH, and one 
issue of a monthly Index Medicus. This third class was unique in 
that they did not receive the traditional library lecture 
explaining the printed resources available to them. They were 
given the most intcmsive instruction on the structure of MeSH and 
the organization and access points of the MEDLINE database. The 
last hour was devoted to "hands-on" instruction but it was held 
in a very small room that had only two microcomputers. This 
arrangement allowed only two students at a time with a maximum of 
six observers. The first attempts including logon connection and 
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display of retrieval took about twenty minutes so that only two 
sets of students, four people, had the hands-on experience. The 
observers changed places so that most members of the class were 
able to see at least a fraction of the full search session 
experience. Since this last hour of instruction was incomplete for 
most of the class, the three hours instruction ti^ne reflects more 
the instructor's involvement than the class participation. 

The fourth class was a graduate course in audiology. There were 
only four students in the class and they, independent of their 
professor and scheduled class time, set up a two-hour lecture with 
the librarian. Their specifications were to learn how to search 
MEDLINE using Grateful Med. The librarian gave the lecture in the 
usual library classroom. The four students used the classroom 
demonstration PC with modem for hands-on experience. The lecture 
on structure of MeSH and Index Medicus was comparable to the ones 
given to the first two classes. 

METHODOLOGY 3VND DATA ANALYSIS 

The XT microcomputer station in the science library had a custom 
log program installed that tracked the total time the students used 
the comput(ir, including search formulation time, on-line time, and 
time to evaluate and print the citations. The log also requested 
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information on name, class, topic, and cost of search, and a 
reminder to fill out the evaluation form. An example of the 
information collected by the computerized log is in FIGURE 1. 
Table 1 tabulates total times recorded on the log . or two of the 
classes, the total cost, and the average cost. Log data for the 
communicative disorder's class was incomplete because the log 
program was not working properly the first weekend when most of the 
students did their searches. For those students the study has dai"a 
on costs from the returned evaluation forms and is reported in 
Table 2. Students were also asked to estimate number of relevant 
citations retrieved and the number of citations they actually 
printed; the results are given in TABLE 2. 

Table 3 reflects use of two groups divided by two criteria, "naive" 
vs. "experienced." In terms of intensity of Grateful Med 
instruction, the communicative disorders class (CD) would be 
considered "naive" since they only received a classroom 
demonstration. The physical therapy (PT) students were considered 
experienced since their instruction included a "hands-on" 
microcomputer lab. Thera were so few returned evaluation forms 
from the audiology (AUD) and health science (HS) classes that most 
times they are excluded from the results. Table 4 was generated 
when a review of the data showed that students who indicated 
previous experience v/ith computers were all in the communicative 
disorders class. Studying just the communicative disorder's class 
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it would seem that the data collected indicates the higher levels 
of satisfaction, the ease in learning and using the system with 
less need of further training is correlated to the amount of 
previous computer experience. 

DISCUSSION 

The majority of students indicated they needed further training. 
Similar results were reported by Ludwig*s study of Medis use [13]. 
In the case of Grateful Med, version 3.0, the students were 
insecure about the proper selection of MeSH terms. Reinforcement 
of this conclusion was gained when near the end of this study on 
Grateful Med, the library received a thirty day trial copy of 
dialog's MEDLINE OnDisc product. Students who had previously used 
Grateful Med commented on how much they liked to choose terms from 
the DIALOG software 'r. dictionary over the Grateful Med's MeSH. The 
students much preferred the way DIALOG shows retrieval for each 
term, gives cross references for MeSH, and lists retrieval for 
MeSH terms with subheading (s) . 

When this study was undertaken it was realized that the likelihood 
of using Grateful Med as a sole research tool in a busy reference 
environment was highly improbable. Several factors were 
considered. Primary was the not always reliable telephone lines. 
Other factor;j were the downtime of the NLM computer, and nhe 

10 



ERLC 



11 



Henry 

uncertainty of the costs involved offering an online service as a 
research tool for students. However, the Grateful Med study did 
show that students frequently liked to search previous years. 
Figure 2 snows the distribution of student usage of MEDLINE files 
by years. [FIGURE 2] An interesting sideline, however, is that once 
the MEDLINE CD~P A was installed which offered 1987 with the option 
of also switching to the current year (Jan. -June 1988), most of the 
students were happy with the retrieval from 1987. The CD~ROn was 
received at the end of the semester so it is doubtful whether these 
students are a representative sample of the research activity which 
occurs earlier in the term. However, their use complies with 
Capodagli's obrervation of patrons' use of Compact Cambridge as 
"fulfilling a basic need — for a few good recent articles" aud 
Glitz 's report that only a few patrons requested more years. 
[14,15] 

A study of the computerized log in Figure 1 indica'cos the stu.^ents 
were rarely online longer than a few minutes, but occupied the 
micrccoiiiputer station for much longer periods of time. Although 
the average costs from the evaulation forms do not exactly match 
the averages from the computerized logs, the data from the Physical 
Therapy class with about the same number of people reporting are 
fairly close. Any projection of costs for a Grateful Med search 
station in an academic environment would have to rely on students 
continuing to selectively print rele^^^wt citations from the 
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retrieval thereby occupying a search station for greater lengths of 
time. 

The high percentage of utre of MEDLINE back files indicates that 
students involved in serious research would access the older 
literature. Although MEDLINE on CD-ROM is also enonp^usly 
successful, students who cwn their own computers or will soon leave 
the academic environment are very interested in learning how to use 
Grateful Med. Twenty- five students indicated they would obtain 
their own NLM passwords, and sixteen planned to purchase the 
Grateful Med software. 

CONCLUSJ 

After studying the use made of Grateful Med by a variety of 
students, with and wi^ o:^t previous computer experience, and with 
varying degrees of instruction in actual use of Grateful Med, 
several observations are appropriate. Almost all students were 
interested in learning how -co use the software and found it easy to 
learn. Any prior experience, either in general computer use, or 
specifically. Grateful Med use, increased the level of 
satir.f action. Students were able to use the Grateful Med software 
effectively. Twenty-five students indicated they would acquire 
their own access codes. Further research on the use of Grateful 
Med as an adjunct to a current MEDLINE CD-ROM product would study 
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students' need for older literature and their willingness to learn 
and use different search protocols. 
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TABLE 2 

Comparison ot tlrg Pour Classes (Evaluation Forms) 
TOTAL [68] CD[34] PT[10] AUDC2] HS[4] 

% 

relevant 62% [43] 59. 5% [28] 49% [8] na na 

% 

printed 65%t66] 76.7%[34] 59%[9] 89%[2] 21% 

average 

cost 3.59[67] 3.26[33] 3.29[10] 4.44 5.79[4] 
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TABLE 3 

Comparison of Experienced vs Naive Computer Users (Evaluation Forms) 
Computer Accesa [25] ho Computer Access [40] PT class[10]CD class [34] 



Require 
further 
training 

easy 
to 

learn 

easy 

to 

use 



52% 



95.4% 



9°% 



67% 



86.8% 



92. 1% 



30% 



lOOi'j 



90% 



67.6% 



91.2% 



94.1% 
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TABLE 4 



Comparison of Computer Experienced vs. Naive (CD class, Evaluation Forms) 

Require Further training Easy to Learn Easy to Use 

Computer Access 54.5% 100% 100% 

No Computer Access 75% 85% 90% 
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FIGURE 1 



Example of Information Recorded by Computerized Log 



Logon 


Logon 








Access 


Date 


Time 


Lib 


Patron 


Course 


Topic 


Time 


Cost 














— — 

02/16/88 


15:40:14 


mh 


envoldson 


cd573 


aphasia 


00:32:59 


$4 .88 














02/23/88 


14:15:49 


mh 


weismann 


cd574 


otitis 


00:15:20 


$1.82 














02/25/88 


14:09:25 


mh 


strong 


hs497 


knees 


01: 14:05 


$5.46 














02/25/88 


15:25:13 


mh 


spalding 


hs497 


running&knee to 


00:36:33 


$2.78 














03/02/88 


15:49:25 


mh 


burkhart^ste 


hs59 4 


juvenile deling 


00:33:06 


$3.26 














03/06/88 


11:25:11 


mh 


low/ sheryl 


hd594 


cerebral palsy 


01:47:16 


$8.45 














03/09/88 


13:51:34 


mh 


f arrenkopf 


hs497 


aerobics 


00:31: 16 


$1.46 














03/09/88 


14:28:32 


mh 


newton 


hs497 


wound therapy 


00:29:54 


$1.25 














03/09/88 


15:01:44 


mh 


roller 


hs497B 


aerobic 


00:24:23 


$3.05 














03/11/88 


13:54 : 00 


mh 


newton, p 


hs497b 


wounds 


00:28:44 


$3.28 














03/12/88 


09:44 : 08 


mh 


heine 


hs497 


aids attitudes 


00:10:55 


$0 .55 














03/12/88 


10:22:08 


mh 


heine, elina 


hs497 


aids Schealth wo 


00:19:06 


$1.52 














03/17/88 


17:44:36 


mh 


taliaf erro^m 


Cd574 


cohlea implants 


00:37:45 


$3.22 














03/25/88 


13:52:42 


mh 


weisman,holl 


cd57 4 


audiomentry 


00:49:34 


$5.67 














03/31/88 


14:59:07 


mh 


nikki washla 


Cd573 


language norms 


00:23:05 


$2.62 














03/31/88 


15:25:15 


mh 


mosesson, j 


h:,497 


spinal atrophy 


01:08-25 


$4.51 














04/07/88 


14:16:34 


mh 


guiteras^p 


hs497 


brainstemenceph 


00:13:53 


$1.08 














04/29/88 


12:27 :04 


mh 


jaeger^gretc 


Cd573 


early int. deaf 


00:17:52 


$3.39 
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FIGURE 2 
YEARS OF FILES SEARCHED 



lYEARS SEARCHED! 



5.ti% 



3U5H 




2M 



I tilediine 
H 1983-88 
S 1980-88 
on 1977-88 
SS 1973-88 
95! 1966-8e 
H Not Given 
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EVALUATION FORM 

SEARCHING NATIONAL LIBRARY OF MEDICINE'S 
MEDLINE OR HEALTH DATABASES 
USING 

GRATEFUL MED SOFTWARE 

Please answer the questions belo\7*. Circle One 

1. Is this the first time you have performed Yes Nc 

a computer search yourself? If you 
circled No, what other search systems have 
you used? 



2. Was this search system easy to learn? Yes No 

3. Did you read the instructions? Yes No 

Did you use the online help instructions? Yes No 

Comment on any weaknesses or omissions in 
either the written or online instructions 




3. Was this search system easy to use? Yes No 

4. Were you satisfied with the results? Yes No 

(1) How iiiany citations did you retrieve? 

(2) How many of those were relevant? 

(3) How many citations did you print? 

{ 'i ) L) d you ask for abstracts? Yes No 

(5) Which database did ycu search? MEDLINE HEALTH 

(6) What years did you search? 

(7) How much did your search(s) cost? (give total) 

Use log, number D. on other side, to record each search 

5. Did you consult the printed indexes ( Jndjex M^ dicus> 
tiospjtal Literature Index ) before doing the 
computer search? Yes No 
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Did you consult Irkdi.cal Subject H^^rjinr-^ (M^)o yes No 
If you circled Yes. did you choose MeSH headings 

(1) Using the FlO function key Yes No 

(2) Typed them in using a forward slash (\) Yes No 

Would you use this system again? Yes No 

{9! i^J^^' "^^^ "PP^^ password? Yes No 

(2) Will you purchase the Grateful Med software 
from NTIS? 

(3) Do you think you require further training? Yes No 
14 J Do you own or have access to an IBM pc (or 

PC compatible) with modem? Yes No 

P?ease describe your search topic (Very important to answer this) 



Circ"2 one: CSUN Faculty 

CSUN Graduate School Student 

CSUN Undergraduate Student 
Department: 

Your Name: 
Date and Time: 

Record the cost, number of citations retrieved and years searche 

Ddtaba^ qq S t citations retrieved 

Medline 



Health 
Additional Comments? 
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